Statistical analysis of the performance of four Apache Spark ML algorithms

Feature selection (FS) techniques generally require repeatedly training and evaluating models to assess theimportance of each feature for a particular task. However, due the increasing size currently availabledatabases, distributed processing has become necessity many tasks. In this context, Apache SparkML library is one most widely used libraries performing classification other tasks with largedatasets. Therefore, knowing both predictive performance efficiency its main algorithms beforeapplying FS technique crucial planning computations saving time. work, comparativestudy four Spark ML carried out, statistically measuring execution times andpredictive power based on number attributes from colon cancer database. Results were analyzed, showing that, although Random Forest Na¨ıve Bayes are shortest times, Support Vector Machine obtains best power. The study these interesting as they applied in different problems, such pathologies epigenomic data, image classification, prediction computer attacks network security among others.